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ARTICLEINFO ABSTRACT 


Keywords: 


The use of video and paper-based materials is commonly widespread in foreign language learning (FLL). It is well 
established that the level of acceptance of these materials influences learning outcomes, but there is lack of 
evidence regarding the use and related impact of videos generated by artificial intelligence (AI) on these aspects. 
This paper used linear mixed models and path analysis to investigate the influence of student acceptance of AI- 
generated short videos on learning outcomes compared to paper-based materials. Student acceptance was 
assessed based on perceived ease of use (PEU), perceived usefulness (PU), attitude (A), intentions (I), and 
concentration (C). The results indicate that both Al-generated short videos and paper-based materials can 
significantly enhance learning outcomes. Al-generated short videos are more likely to be accepted by students 
with lower pre-test scores and may lead to more significant learning outcomes when PEU, PU, A, I and C are at 
higher levels. On the other hand, paper-based materials are more likely to be accepted by students with higher 
pre-test scores and may lead to more significant learning outcomes when PEU, PU, A, I and C are at lower levels. 
These findings offer empirical evidence supporting the use of Al-generated short videos in FLL and provide 
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Learning outcomes 

AI videos 

Short videos 

Foreign language learning 


suggestions for selecting appropriate learning materials in different FLL contexts. 


1. Introduction 


According to the Cognitive Theory of Multimedia Learning, people 
learn better from text and pictures than from text alone (Mayer, 2020). 
In comparison to text and audio, the use of video in learning environ- 
ments offers students a more comprehensive and engaging approach to 
understanding learning materials (Hung, 2016). Among these 
video-based materials, short videos have emerged as an innovative 
pedagogical approach that have been gaining popularity across various 
educational settings in recent years (Kruger & Doherty, 2016). This 
growth can be largely attributed to the widespread usage and daily 
reliance on digital technologies and social media platforms (Jong et al., 
2020; Kaye et al., 2022; Zahirovié Suhonjié et al., 2019). 

Evidence suggests that regardless the video length, students' average 
engagement time does not exceed 6 min, with the highest engagement 
observed when videos are only kept under 3 min (Guo et al., 2014). This 
trend is supported by feedback from students, who prefer shorter videos 
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and are strongly interested in specific video formats, such as practical 
demonstrations (Choe et al., 2019). Furthermore, recent investigation 
on videos containing scientific information, particularly on representa- 
tive social media platforms like TikTok, reveal that most of these videos 
are less than a minute and their contribution to knowledge construction 
is beginning to be studied (Nguyen & Diederich, 2023). 

Therefore, shorter videos are attractive not only because of their 
brevity but also because they can be better planned than longer videos 
(Guo et al., 2014). There are already some benefits that highlight the 
importance of incorporating short videos into educational practice. For 
example, short videos can improve students' learning outcomes and 
their desire to revisit the content (Carmichael et al., 2018), and are more 
effective than longer videos in managing and reducing students’ 
cognitive load (Lee et al., 2021). 

Most studies on short videos have focused on contrasting them with 
longer video formats (Afify, 2020; Manasrah et al., 2021; Yu & Gao, 
2022), with few studies examining the effect of short videos on learning 
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outcomes and student acceptance in comparison to paper-based mate- 
rials. The existing ones yield mixed results regarding the effectiveness of 
short videos versus paper-based materials (Tarchi et al., 2021; Van Der 
Meij & Van Der Meij, 2014). 

Recent advances in generative artificial intelligence (AI) technolo- 
gies have expanded capabilities into areas previously considered unat- 
tainable, including the production of short videos (Leiker et al., 2023). 
Current AI for video generation facilitate the conversion of pre- 
sentations, websites, and text content into video format (UNESCO, 
2023). This means that videos can be generated with the same content as 
paper-based learning materials easily. However, the use of AI-generated 
short videos in educational settings is still in its initial stages. The 
learning outcomes and acceptance of these videos in foreign language 
learning (FLL) remain unclear. 

Therefore, given the rapid development of AI technology, which 
enables the rapid conversion of paper-based content into video format, 
and the ongoing debate about the effectiveness of short videos versus 
paper-based materials, a comparison between AI-generated short videos 
and paper-based materials may have considerable empirical signifi- 
cance. Furthermore, the extent to which the acceptance of AI-generated 
short videos correlates with improved learning outcomes, particularly in 
the context of FLL, remains unclear. Consequently, this study aims to 
address this gap by comparing the learning outcomes and student 
acceptance of Al-generated short videos with those of paper-based 
materials. 


1.1. Learning outcomes and acceptance of short video compared to paper- 
based materials 


Video and paper are two commonly used materials in learning, and 
their impact on learning outcomes and acceptance has been consistently 
compared. Tarchi et al. (2021) compared three learning conditions 
differing for modality: narrated video versus subtitled video versus text 
with graphics, and assessed their impact on students' learning outcomes. 
The video was created by an educational publisher and had a duration of 
6 min. The results indicated that immediate and delayed comprehension 
were similar across all conditions. Conversely, Van Der Meij and Van Der 
Meij (2014) analysed the presentation of software instructions via video 
and via paper. They compared four tutorial configurations: paper-based, 
paper preview and video procedure, video preview and paper proced- 
ure, and video-based. The videos were self-made and lasted for 2 min. 
The results of the study revealed significant and substantial improve- 
ments across all conditions. However, participants in the paper-based 
condition exhibited lower performance compared to the other condi- 
tions. Instead, video worked better for preview tasks and favoured the 
presentation of software instructions. 

In the field of FLL, Redzioch-Korkuz (2019) conducted a study on the 
impact of using subtitled short films versus paper-based materials on 
vocabulary acquisition. The findings showed that while there were slight 
differences in the overall performance, these did not reach statistical 
significance. Nevertheless, students in the video group exhibited a 
greater enthusiasm for learning, as they were able to facilitate easier 
retention. Zhang et al. (2021) conducted a study to explore the differ- 
ences in learning outcomes among second language learners who used 
short video clips or paper-based materials for vocabulary and phrase 
acquisition. The results revealed that students made significant progress 
and showed a positive attitude towards learning when using video-based 
materials. Subsequently, Huang et al. (2022) investigated students' 
perceptions of using multimodal materials compared to paper-based 
materials for foreign language phrases learning. The videos were seg- 
ments from films and television programmes. The findings indicated 
that both groups demonstrated substantial improvements, with students 
exposed to video-based materials demonstrating better performance and 
a positive attitude towards learning. 

The multimodal nature of short videos fosters effective and engaging 
learning experiences (Choe et al., 2019). Short videos not only can 
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increase viewer engagement but also enhance the likelihood of repeated 
usage (Carmichael et al., 2018). Furthermore, short videos have the 
capacity to simulate professional contexts (Ogea, 2020), and can be 
employed to enhance motivation and participation in FLL 
(Larreta-Azelain & Monje, 2016). 

However, Jarvis and Szymczyk (2009) investigated students’ atti- 
tudes towards learning grammar in a self-paced environment and their 
preferences for learning materials. They found that students generally 
preferred to use paper-based materials. Isla-Montes et al. (2022) indi- 
cated that when comparing the effectiveness of the writing task in paper 
and digital formats, paper-based materials were found to be more ad- 
vantageous in enhancing students’ language proficiency. While digital 
materials facilitated student interaction, they may have not effectively 
supported the development of students’ communicative skills. Yu et al. 
(2022) also concluded that participants exhibited enhanced reading 
comprehension accuracy when reading from printed paper compared to 
reading from a mobile device. 

Consequently, it is challenging to reach a definitive conclusion 
regarding the comparative merits of short video and paper-based ma- 
terials in diverse contexts and for various purposes. Different studies 
focus on comparing these two types of materials, but none uses AI- 
generated short videos as a video mode. 


1.2. Learning outcomes and acceptance of AI-generated videos compared 
to human videos 


Generative AI represents a form of AI technology that is capable of 
autonomously generating content encompassing all forms of human 
cognitive representations based on prompts provided within natural 
language dialogue interfaces (UNESCO, 2023). Its rapid advancement 
underscores its role in supporting learning and well-being 
(Pataranutaporn et al., 2021). At the present time, video-generating AI 
technologies can transform a variety of content, including presentations, 
websites, and textual content, into video formats effortlessly (UNESCO, 
2023). A recent study compared the impact of AI-generated videos and 
human videos on learners’ content acquisition and learning experiences 
(Leiker et al., 2023). The findings revealed no significant differences 
between the two conditions regarding learning outcomes and partici- 
pants' perceptions of the videos. Consequently, tasks that were previ- 
ously considered to be beyond the capabilities of AI can now be 
accomplished. 

Prior to the integration of AI-generated speech and avatars in videos, 
a series of studies have been conducted to compare the effectiveness of 
virtual speech and avatars. Craig and Schroeder (2017) compared pre- 
senters with modern speech engine and human voice. The results indi- 
cated that there were no significant differences between the two in 
facilitating learning. It was emphasized that while learners may perceive 
differences in the voices of the presenters, these distinctions are insuf- 
ficient to impact their trust in the information presented. Their subse- 
quent research demonstrated that, despite participants reporting a 
preference for human voice, no significant differences were evident in 
learning outcomes or cognitive efficiency measures (Craig & Schroeder, 
2019). Similarly, Davis et al. (2019) demonstrated that even in 
non-native contexts, modern computer-generated speech can be as 
effective as human voice in knowledge retention. Furthermore, Vallis 
et al. (2023) discovered that AI avatars are perceived as effective tools 
for content delivery, with students demonstrating a high level of satis- 
faction and at times even regarding them as high-quality delivery 
mechanisms. 

Although some studies have showed that virtual agents may produce 
comparable learning outcomes, they may also influence learners’ trust 
and perceptions in disparate ways (Chiou et al., 2020). It is crucial to 
acknowledge that the efficacy of human voices may not necessarily 
exceed that of modern computer-generated speech if auditory elements 
are not effectively conveyed in accordance with audience needs (Davis 
et al, 2019). Conversely, AI instructors are capable of consistently 
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delivering effective instructional content, thereby avoiding the potential 
for emotional fluctuations (Kim et al., 2022). 

Therefore, given the potential for Al-generated videos to achieve 
comparable learning outcomes and acceptance as human videos, and the 
capability of AI technology to swiftly convert paper-based content into 
video format, Al-generated videos hold notable potential for use in a 
wide range of educational settings. Consequently, a comparison between 
Al-generated short videos and paper-based materials may have consid- 
erable empirical significance. Moreover, the relations between the 
acceptance of Al-generated short videos and enhanced learning out- 
comes, particularly in the context of FLL, remains unclear and warrant 
further research. 


1.3. The Technology Acceptance Model in education 


The Technology Acceptance Model (TAM) served as a theoretical 
framework employed to elucidate and predict the acceptance of novel 
technologies by individuals (Davis et al., 1989). This framework has 
been repeatedly affirmed as suitable for delving into the variables 
influencing technology acceptance within educational research. For 
instance, Liu et al. (2009) combined the classic TAM with flow theory 
and media richness theory. Students interacted with one of three com- 
binations: text, streaming audio, or streaming video. Their findings 
confirmed that perceived ease of use predicts perceived usefulness, 
while perceived usefulness and attitudes predict intention. Additionally, 
richer content combinations were associated with increased attention 
levels, though the correlation with perceived usefulness showed mixed 
results. 

Huang et al. (2012) employed the TAM to explore students’ per- 
ceptions of a self-developed English vocabulary learning system. Their 
results indicated that active students were interested in perceived use- 
fulness, while passive students were interested by perceived ease of use. 
Tsai (2015) used the TAM to evaluate the impact of course management 
system on English writing teaching, revealing positive outcomes mainly 
attributed to technical support, perceived usefulness, perceived ease of 
use, and attitude. Nagy (2018) extended the TAM to examine student 
video usage determinants and learning satisfaction. Results showed that 
perceived usefulness, attitude, and internet self-efficacy directly 
impacted video usage, while learning satisfaction was influenced by 
learner interaction, perceived ease of use, and learning performance. 
Moreover, video usage significantly affected learning performance and 
satisfaction. Pal and Patra (2021) proposed an integrated the TAM to 
understand students’ adoption of digital technology for learning. Their 
study revealed a strong correlation between video learning success, 
students' perception, system usage, and individual characteristics. 

Against this background, the application of the TAM proved suitable 
to analyse students' acceptance of AI short videos, thereby providing 
insights into the factors that facilitate or hinder their acceptance in the 
context of this study. 


1.4. Study aim 


The study aims to compare the learning outcomes and student 
acceptance of AI-generated short videos with paper-based materials in 
the context of FLL with the following research question: How does stu- 
dent acceptance of Al-generated short videos influence learning out- 
comes in FLL compared to paper-based materials? 

Additionally, the study tests six hypotheses based on Liu et al. 
(2009). H1: perceived ease of use (PEU) is positively associated with 
perceived usefulness (PU); H2: PEU is positively associated with atti- 
tudes (A); H3: PU is positively associated with A; H4: PU is positively 
associated with intentions (I); H5: A is positively associated with I; H6: 
Concentration (C) is positively associated with I. 
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2. Methods and materials 
2.1. Participants and procedure 


This is an exploratory study comprising 20 students who were 
studying Portuguese as a foreign language at the B1 proficiency level ina 
mid-to-large public Higher Education Institution on the central coast of 
Portugal. These participants were randomly divided into two groups: the 
video group (n — 10) and the paper group (n — 10). The video group 
watched AI-generated short videos, while the paper group used paper- 
based materials with content that matched the videos. Participation in 
the study was voluntary, anonymous, and fully consensual, with 
approval obtained from the Ethics and Deontology Committee of the 
University. Each participant was exposed to the experimental materials 
four times. The experimental instrument included a pre-test, a post-test, 
and a survey of acceptance. Data collection took place in the academic 
year 2023/24, during regular classes in two B1 level courses. 


2.2. Learning materials 


The learning content was taken from a textbook on European Por- 
tuguese oral expression (Boléo & Dourado, 2019). Two experienced 
Portuguese (as foreign language) teachers identified specific chapter 
topics according to the students' proficiency level. Four paper-based 
materials were produced, in accordance with the B1 level content out- 
lined in the textbook. Each material contained 5 to 6 everyday expres- 
sions used in European Portuguese. 

Subsequently, four paper-based materials were converted into short 
video of approximately 40 s, using the Synthesia platform. This platform 
allows for the generation of videos featuring dynamic characters 
through script inputs and supports the use of these videos in educational 
research. Given the effectiveness of multimedia learning principles in 
video design (Mayer, 2021), these principles were used as the basic 
framework for video production. For instance, most harmonious com- 
bination of female avatar and voice were selected, following the prin- 
ciples of voice, image, and embodiment. The video was then edited to 
include subtitles, prompts, background standardisation and pacing ad- 
justments. These modifications addressed concerns related to coherence, 
signalling, spatial contiguity, segmenting, and multimedia principles. 
The primary goal of these improvements was to reduce the potential 
impact of external variables on video design, thereby ensuring absolute 
consistency across all videos beyond the learning content. 


2.3. Technology acceptance model 


Considering that the objective of this paper was not to identify new 
external variables to extend the TAM, we used the framework previously 
expanded by Liu et al. (2009) to compare the factors influencing the 
acceptance of videos and paper materials. This integrated framework 
measured participants' PEU, PU, A, I and C, using a questionnaire with 
17 seven-point Likert-scale items ranging from ‘strongly disagree’ (1) to 
‘strongly agree’ (7) (see Table 1). The questionnaire demonstrated good 
and excellent global reliability with our sample, with a Cronbach's alpha 
coefficient of PEU (a = .929), PU (a = .959), A (a = .897), I (a = .828) 
and C (a = .926) (Nunnally & Bernstein, 1994). 


2.4. Synthetic dataset generation 


The generation of synthetic data involves extracting information 
from actual respondent datasets and reconstructing the outcomes into a 
collection of synthetic datasets for public use (Raghunathan, 2020). In 
the field of education, synthetic data is used to quickly create complex 
and stratified datasets, addressing ethical and privacy risks in service 
development processes (Berg et al., 2016). Furthermore, synthetic data 
can be employed to substitute missing data or facilitate disambiguation 
processes (Ferreira et al., 2012), thereby enabling the generation of 


Y. Zhang et al. 


Table 1 
Scales of student acceptance. 

Scale Item 

name 

PEU1 It is easy for me to remember how to carry out tasks using the learning 
material. 

PEU2 I believe that it is easy to get the learning material to do what I want it 
to do. 

PEU3 My interaction with the learning material is clear and understandable. 

PEU4 Overall, I believe that the learning material is easy to use. 

PUI Using the learning material improves my schoolwork performance. 

PU2 Using the learning material improves my productivity. 

PU3 Using the learning material enhances my effectiveness as regards 
schoolwork. 

PU4 Overall, I find that using the learning material is useful in my 
schoolwork. 

Al I like using the learning material. 

A2 The learning material is fun to use. 

A3 The learning material provides an attractive working environment. 

n I intend to completely switch over to the learning material. 

I2 I intend to increase my use of the learning material in the future. 

I3 Assuming that I have access to the learning material, I intend to use it. 

C1 I was absorbed intensely in the learning material. 

C2 My attention was focused on the learning material. 

C3 I was deeply engrossed in the learning material. 


larger datasets than those feasible with real data alone. This allows for 
the identification of patterns that would otherwise remain obscured in 
the original dataset (Eno & Thompson, 2008). 

In this study, synthetic data is used to address limitations in sample 
sizes. Nufberger et al. (2021) evaluated the effectiveness of three 
commonly used generative modelling methods: Variational Autoen- 
coders (VAE), Deep Boltzmann Machines (DBM), and Generative 
Adversarial Networks (GAN), under conditions of limited sample sizes. 
The results suggest that VAE and DBM are more suitable for handling 
small sample sizes compared to GAN. Importantly, synthetic data can 
exhibit statistically similar characteristics to real data and adheres to 
any logical validity constraints imposed on the real data (Raghunathan, 
2020). 

Given the limited sample size in our study, a VAE (Kingma & Well- 
ing, 2014) was employed to generate synthetic data from the collected 
measurements. We standardized the numerical data using the Stand- 
ardScaler (Pedregosa et al., 2011) and then converted it into PyTorch 
tensors (Paszke et al., 2019). The encoder, a neural network, produced 
mean and log variance outputs. The decoder, with three layers, used a 
sigmoid function to generate data. During training, two distinct VAE 
models were trained, one for the video group and the other for the paper 
group. Throughout the training process, we iteratively adjusted the 
models to minimize the disparity between the generated data and the 
original dataset, ensuring the distribution of the new data points 
remained consistent. Finally, 300 synthetic data were generated for each 
of the video and paper groups, and the synthetic dataset for each group 
was formed by combining the original data with the generated synthetic 
data. 


2.5. Data analysis 


To investigate differences in learning outcomes and student accep- 
tance regarding the use of AI-generated short videos versus paper-based 
materials, we used linear mixed models with restricted maximum like- 
lihood estimation (REML) (Littell et al., 2006). The model included a 
random intercept for intercept ID to account for potential variability 
between participants. The total tests were measured by combining the 
scores from the four tests. Additionally, the scores from the first test 
were evaluated separately to ensure an unbiased response, as the total 
tests may have been influenced by carry-over effects. A path analysis 
was also performed using maximum likelihood with robust standard 
errors. The path analysis was based on a synthetic dataset to enhance the 
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statistical inference and maintain the integrity and representativeness of 
the original data characteristics. The purpose of this analysis was to 
clarify the direct and indirect effects among PEU, PU, A, I, and C, and to 
identify the main paths that influence intention to use within each 
group, providing a comprehensive understanding of the variables that 
drive student acceptance. 


3. Results 


A mixed model was conducted with the first test and the total tests as 
dependent variables and condition (video-based and paper-based ma- 
terials) and moment (pre-test and post-test) as independent variables. 
The main effect of the moment was found to be highly significant for the 
first test (F(1,9) — 36.0, p « .001) and the total tests (F(1,9) — 66.3, p « 
.001) in the video group. This was also highly significant for the first test 
(F(1,9) = 57.5, p « .001) and the total tests (F(1,9) = 111, p < .001) in 
the paper group. These results indicate that both materials significantly 
enhanced learning. Additionally, the interaction effect between ‘con- 
dition**moment’ was not significant for either the first test (F(1,18) = 
0.053, p — .820) or the total tests (F(1,18) — 0.425, p — .523). This in- 
dicates that the difference between the pre-test and post-test did not 
significantly vary between the two materials. 

The student acceptance variables (PEU, PU, A, I and C) were then 
included as covariates in the previous model. The main effects of PEU, 
PU, A, I and C were found to be not significant for both the first test and 
the total tests. Similarly, the two-way interaction effects including 
‘condition**moment’ and ‘condition**PEU/PU/A/I/C’ did not result in 
any significant interactions. However, there were exceptions within the 
three-way interactions ‘condition**moment*PEU/PU/A/I/C’, indi- 
cating complexities that require further investigation (See Table 2). 

The first test results showed significant interactions for PEU, PU, A 
and C, while the variable I reached marginal significance. In relation to 
the total tests results and considering carry-over effects, there was a 
general decrease in the significance levels for all variables, although 
significant interactions were still observed for PEU and C, PU and I 
showed marginal significant interactions. However, A did not show 
significant interactions. It is important to highlight that despite the 
absence of significant differences in learning outcomes between the two 
materials, incorporating student acceptance variables as covariates in 
the analysis significantly influenced the variance in learning outcomes. 

The distribution of learning outcomes between the two materials 
presents different mean levels of student acceptance (see Fig. 1). This 
visualization was conducted for both the first test and total tests. In both 
scenarios, it was observed that within the Mean-1-SD column, the use of 
paper-based materials demonstrated a more inclined trend in compari- 
son to the Al-generated short videos. This indicates that paper-based 
materials may lead to more significant learning outcomes when PEU, 
PU, A, I and C are at lower levels. Conversely, within the Mean--1-SD 
column, AI-generated short videos demonstrated a more inclined trend 
in comparison to paper-based materials. This suggests that AI-generated 
short videos may lead to more significant learning outcomes when PEU, 
PU, A, I and C are at higher levels. 

Fig. 2 provides an explanation for the observed results. In the video 
group, students with lower acceptance levels (blue line) scored higher in 
the pre-test for both the first test and the variables A, I, and C of the total 
tests. In contrast, students with higher acceptance levels (yellow line) 
reached lower pre-test scores, but the more inclined trend of the yellow 
line indicates their better learning outcomes. The results can be 
explained by the fact that AI-generated short videos may lead to more 
significant learning outcomes when PEU, PU, A, I and C are at higher 
levels. 

In contrast, students in the paper group with lower acceptance levels 
(blue line) scored lower in the pre-test for both the first test and the total 
tests, while those with higher acceptance levels (yellow line) scored 
higher in the pre-test. The blue lines also indicate a more inclined trend, 
suggesting better learning outcomes for these students. The results can 
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Table 2 


Three-way interaction results. 
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First test 


F 


Num df 


Total tests 


F 


Num df 


condition ** moment * PEU 
condition ** moment * PU 
condition ** moment * A 
condition ** moment * I 
condition ** moment * C 


19.679 
5.882 
4.643 
4.049 
8.565 


PRP RP Re 
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16 
16 
16 


7.688 
4.418 
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3.828 
6.380 
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Note: PEU (perceived ease of use), PU (perceived usefulness), A (attitude), I (intentions), C (concentration). 
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Fig. 1. Differences of learning outcomes divided by student acceptance variables. Note: @ (video group), @ (paper group), PEU (perceived ease of use), PU 


(perceived usefulness), A (attitude), I (intentions), C (concentration). 
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Fig. 2. Differences of learning outcomes according to learning materials. Note: @ (Mean-1-SD), 


(perceived usefulness), A (attitude), I (intentions), C (concentration). 


be explained by the fact that paper-based materials may lead to more 
significant learning outcomes when PEU, PU, A, I and C are at lower 
levels. 

Therefore, in addition to the fact that students with lower pre-test 
scores may achieve better learning outcomes, the results also sug- 
gested that Al-generated short videos are more likely to be accepted by 
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students with lower pre-test scores, whereas paper-based materials are 
more likely to be accepted by students with higher pre-test scores. 

The linear mixed model results did not show any significant in- 
teractions for the variable I in both the first test and total tests. Addi- 
tionally, the variable A did not even reach marginal significance in the 
total tests. To further explore the intrinsic relationships among 
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Fig. 3. Path analysis of PEU, PU, A, I and C for the two groups. Note: ***p<. 001, **p<. 01, *p<. 05, > significant path, -> non-significant path. 


acceptance variables and their impact on A and I, a synthetic dataset was 
used for path analysis (see Fig. 3). It is important to note that the path 
analysis aimed to investigate previously established paths (Liu et al., 
2009) rather than validate new paths. 

The path analysis results indicate that neither group was able to fully 
replicate all the proposed hypotheses, especially in relation to variables 
A and I. The variable A in the video group was not adequately explained 
by its predictors (R2 — 0.114), particularly by the PEU (f — —0.145, p — 
.484). Additionally, the variable A had only a marginal influence on I (f 
— 0.442, p — .057). The results were consistent with previous findings 
from the linear mixed model, where the variable A showed no significant 
or marginally significant effects in the three-way interaction of the tests, 
distinguishing it from other variables. This suggests that other variables, 
not considered in this study, may have a greater influence on A. 

Subsequently, an analysis of all paths leading to the variable I was 
conducted. Table 3 demonstrates that both groups had only one effective 
path leading to I. For the video group, the path was from PEU to PU toI 
(p — 0.283, p — .020). This relation suggests that student's perceived 
ease of use of Al-generated short videos enhances their perceived 
learning outcomes, thereby promoting the intention to use videos. This 
finding supports the results ofthe linear mixed models, indicating that at 
higher levels of acceptance, AI-generated short videos may lead to more 
significant learning outcomes compared to paper-based materials. 

However, the effective path for the paper group was from PEU to A to 
I (p = 0.319, p = .038). This suggests that the ease of use of paper-based 


subsequently influencing their intentions to use them. However, PU 
cannot predict A (f = 0.346, p = .181) or I (f = —0.093, p = .677). PU 
refers to learners’ perception of the materials’ ability to enhance 
learning outcomes. This indicates that even if students believe paper- 
based materials can improve their performance, this belief does not 
necessarily result in a more positive attitude towards the materials or a 
stronger intention to use them. 


4. Discussion 


The present study aimed to answer the following research question: 
How does student acceptance of Al-generated short videos influence 
learning outcomes in FLL compared to paper-based materials? 

The results found significant main effects related to the test moment 
for both the first test and the total tests. These results confirmed that 
both AI-generated short videos and paper-based materials significantly 
enhanced learning, in line with the findings of previous studies on the 
impact of short videos and paper-based materials on learning (Huang 
et al., 2022; Tarchi et al., 2021; Zhang et al., 2021). The results further 
supported the findings that Al-generated videos can foster learning 
outcomes (Kim et al., 2022; Leiker et al., 2023; Vallis et al., 2023). 

However, the interaction between material type and moment 
showed no significant difference for either the first or the total test, 
indicating that the two materials did not differ significantly in terms of 
enhancing learning outcomes. This was in line with Clark (1983) who 


materials enhances students’ positive attitudes towards them, argued that the educational media themselves do not directly influence 

Table 3 

Paths to intentions. 
Label Description Estimate SE 95% Confidence Intervals p pA P 

Lower Upper 

Video1 (PEU > PU s A>) 0.144 0.106 —0.064 0.352 0.112 1.358 174 
Video2 (PEU > PU >I) 0.364 0.157 0.057 0.671 0.283 2.322 020 
Video3 (PEU > A > I) —0.082 0.129 -0.335 0.170 —0.064 —0.639 523 
Video4 (PU2A-I) 0.168 0.118 —0.063 0.398 0.180 1.425 154 
Paper1 (PEU2PU2A-I) 0.267 0.226 —0.176 0.710 0.204 1.183 237 
Paper2 (PEU > PU => I) —0.100 0.251 —0.593 0.392 —0.077 —0.399 690 
Paper3 (PEU2A-]I) 0.417 0.201 0.023 0.811 0.319 2.072 .038 
Paper4 (PU>A>=>)D 0.449 0.368 —0.273 1.171 0.247 1.220 223 


Note: PEU (perceived ease of use), PU (perceived usefulness), A (attitude), I (intentions), C (concentration). 
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learning outcomes, but that it is the teaching strategies and content that 
play a crucial role. Thus, while the positive influence of multimedia on 
FLL is widely recognized (Lee & Mayer, 2018), there are still conflicting 
results regarding the effectiveness of different input modalities (Zhang & 
Zou, 2022). Previous studies have yielded mixed results regarding the 
comparative effectiveness of short videos and paper-based materials 
(Tarchi et al., 2021; Van Der Meij & Van Der Meij, 2014). This situation 
has also been observed in the context of FLL (Huang et al., 2022; Red- 
zioch-Korkuz, 2019; Zhang et al., 2021). However, it is important to note 
that these studies separate the influence of learning outcomes and stu- 
dent acceptance of short videos and paper-based materials. They do not 
correlate the acceptance of short videos with improved learning 
outcomes. 

Our results indicated that student acceptance may play different 
roles under different learning materials and influence learning out- 
comes. This is consistent with the findings of previous studies, for 
instance, Tsai (2015) who assessed the impact of course management 
systems on English writing instruction, highlighting effects linked to 
technical support, perceived usefulness, ease of use, and attitude. Nagy 
(2018) investigated the factors influencing student video usage and 
learning satisfaction. The findings indicated that perceived usefulness, 
attitude, and self-efficacy were associated with video usage. Addition- 
ally, learner interaction, ease of use, and performance were identified as 
key factors influencing learning satisfaction. Pal and Patra (2021) also 
revealed strong correlations between video learning success, perception, 
system usage, and individual traits. 

Our results indicated that when levels of PEU, PU, A, I and C are low, 
paper materials may produce more significant learning outcomes; 
conversely, when levels of these variables are high, AI-generated short 
videos may be more effective. We have explained the reason for the 
increased learning outcome, as both levels of acceptance in different 
materials correspond to students with lower pre-test scores. In other 
words, both materials enable students with lower pre-test scores to 
achieve higher learning outcomes. Even students with lower pre-test 
scores, who were initially less willing to use paper materials, did not 
experience a negative impact on their learning outcomes. Therefore, our 
findings support the notion that lower performing students may have 
significant potential for improvement compared to higher performing 
students (Yang & Li, 2018). 

A more important observation is that Al-generated short videos were 
more likely to be accepted by students with lower pre-test scores, while 
paper-based materials were more likely to be accepted by students with 
higher pre-test scores. It should be noted that this comparison did not 
imply that students with higher pre-test scores preferred paper-based 
materials to video ones, but rather that students with higher pre-test 
Scores were more receptive to paper-based materials compared to stu- 
dents with lower pre-test scores. 

Videos are more likely to be accepted by students, as both attitude 
and ability play a crucial role in determining where attention is focused 
(Schmidt, 2010). In our Al-generated short videos, both speech and 
avatar were produced by AI. It is well-established that videos, as a 
multimodal material, engage both visual and auditory channels. 
Therefore, improper design of multimodal elements in videos may lead 
to a higher cognitive load compared to paper-based materials, thereby 
affecting learning outcomes (Mayer, 2020). Within a group of students 
exposed to identical instructional designs, differences in prior knowl- 
edge represent the primary source of cognitive diversity among in- 
dividuals (Sweller, 2024). Thus, while various cognitive elements 
contribute to individual variations, disparities in acquired knowledge 
significantly influence how individuals process information. Conse- 
quently, in contrast to students with higher pre-test scores, those with 
lower pre-test scores face greater demands on their memory retention 
(Sweller, 2024). Therefore, if AI-generated speech and avatars were to 
exert a degree of negative influence on students' cognition, it would be 
unlikely that individuals with lower pre-test scores would exhibit higher 
levels of acceptance or achieve superior learning outcomes. As 
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emphasized in our introduction, tasks previously considered beyond the 
capabilities of AI can now be accomplished effortlessly (Kim et al., 2022; 
Vallis et al., 2023), with Al-generated videos achieving comparable 
levels of learning outcomes and perception as human videos (Leiker 
et al., 2023). 

The path analysis results indicated that, for the video group, the path 
from PEU via PU to I was significant. This relation suggests that 
perceived ease of use of Al-generated short videos enhances students’ 
perceived learning outcomes, which in turn promotes their intention to 
use Al-generated short video. This finding reinforces that AI-generated 
short videos not only facilitate learning outcomes but also influence 
students’ perceptions of their learning outcomes positively. 

An unexpected finding, for the Al-generated videos was that the path 
from PEU to I, mediated by A, was not significant. Furthermore, the 
variable A showed no significant or marginally significant effects within 
the three-way interaction in the tests, setting it apart from other vari- 
ables in the study. These findings challenge the assumption that the 
perceived ease of use of Al-generated short videos necessarily foster 
more positive attitudes or stronger intentions to use. Additionally, it 
implies that other variables, not considered in this study, such as video 
duration, AI speech and AI avatar, may have a greater influence on A. 
However, the current data do not directly prove whether this entails a 
higher or lower level of acceptance, nor is it possible to precisely 
determine the other variables that affect A. These aspects warrant 
further investigation. 

Additionally, for the paper-based materials, the path from PEU via A 
to I was significant. This suggests that the ease of use of paper-based 
materials enhances students’ positive attitudes towards them, subse- 
quently influencing their intentions to use them. Nonetheless, students 
did not necessarily translate their perceptions of the potential learning 
outcomes of paper-based materials into increased intentions to use or 
more positive attitudes towards these materials. Therefore, it is recom- 
mended that teachers not only emphasise the effectiveness of learning 
materials but also focus on the positive acceptance towards using them. 

Finally, we consider the implications of this study for educational 
theory and pedagogy. This study validates the effectiveness of short 
videos and Al-generated ones as pedagogical tools. It suggests that AI- 
generated content can align with multimedia learning theories, 
thereby enhancing our understanding of how AI technologies can 
improve educational practices. Additionally, our findings underscore 
the critical role of students’ acceptance of various types of learning 
materials on their learning outcomes. This emphasizes an important 
pedagogical implication that teachers should consider regarding the 
interplay between the effectiveness of learning materials and student 
acceptance. In other words, instructional strategies should include an 
understanding of how students perceive and interact with learning 
materials, even within an Al-enhanced learning environment. Further- 
more, our study provides empirical evidence supporting the flexible use 
of Al-generated short videos alongside traditional paper-based mate- 
rials, consistent with the theory of differentiated instruction. By 
leveraging this flexibility, teachers can optimize learning outcomes and 
foster more inclusive and effective educational environments. 


4.1. Limitations 


This study has inherent limitations. First, the research sample mainly 
consisted of learners at a lower proficiency level in Portuguese, which 
limits the generalizability of the study’s findings to foreign language 
learners of different proficiency levels and learners in other disciplines. 
Second, the sample size was limited, which may have introduced bias in 
predictive effectiveness, despite the use of synthetic data to compensate 
for statistical inadequacies. 


5. Conclusion 


This study aimed to investigate the influence of student acceptance 
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of AI-generated short videos on learning outcomes in FLL compared to 
paper-based materials. The results indicate that both AI-generated short 
videos and paper-based materials can significantly enhance learning 
outcomes. Al-generated short videos are more likely to be accepted by 
students with lower pre-test scores and may lead to more significant 
learning outcomes when PEU, PU, A, I and C are at higher levels. On the 
other hand, paper materials are more likely to be accepted by students 
with higher pre-test scores and may lead to more significant learning 
outcomes when PEU, PU, A, I and C are at lower levels. Additionally, for 
AI generated short videos, the path from PEU via PU to I was significant. 
For the paper-based materials, the path from PEU via A to I was sig- 
nificant. To improve the external validity of the study, future research 
should include more participants from diverse educational backgrounds 
and proficiency levels. Furthermore, the PEU of Al-generated short 
videos did not result in more positive A. Therefore, it is necessary to 
investigate other variables such as video duration, AI speech and AI 
avatar, that may have a greater influence on A in further studies. In 
summary, this research contributed to the existing literature on AI- 
generated short videos and their integration into educational contexts. 
The empirical evidence presented in our findings supported the use of 
Al-generated short videos in FLL and provided suggestions for selecting 
appropriate learning materials in different contexts of FLL. 
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